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Abstract. We demonstrate through two case studies, one on the p-spin interaction 
model and the other on the random iC-satisfiability problem, that a heterogeneity 
transition occurs to the ground-state configuration space of a random finitc- 
connectivity spin glass system at certain critical value of the constraint density. At 
the transition point, exponentially many configuration communities emerge from the 
ground-state configuration space, making the entropy density s(q) of configuration- 
pairs a non-concave function of configuration-pair overlap q. Each configuration 
community is a collection of relatively similar configurations and it forms a stable 
thermodynamic phase in the presence of a suitable external field. We calculate s(q) 
by the replica-symmetric and the first-step replica-symmetry-broken cavity methods, 
and show by simulations that the configuration space heterogeneity leads to dynamical 
heterogeneity of particle diffusion processes because of the entropic trapping effect of 
configuration communities. This work clarifies the fine structure of the ground-state 
configuration space of random spin glass models, it also sheds light on the glassy 
behavior of hard-sphere colloidal systems at relatively high particle volume fraction. 
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1. Introduction 

Spin glass models denned on finite-connectivity random graphs have two control 
parameters, one is the temperature T and the other is the interaction (or constraint) 
density, defined as the number M of interactions (constraints) versus the number N of 
vertices, a = M/N. Theoretical work of the last ten years [H El [3] have established 
the understanding that, a random spin glass system with many-body interactions 
will experience a clustering transition if either the temperature T is lowered below 
certain threshold value or the constraint density a is increased beyond certain threshold 
value. At the clustering transition, the configuration space of the system splits into 
an exponential number of Gibbs pure states and ergodicity is broken. This transition 
is followed by another phase transition of the configuration space, the condensation 
transition, as T is further lowered or a further increased. At the condensation transition, 
a sub-exponential number of large Gibbs states start to dominate the configuration 
space and hence the equilibrium property of the system [3]. These properties were first 
observed in fully-connected mean-field spin glass models [I]. Whether they are also valid 
in .D-dimensional real- world spin glass systems (D = 3 or D = 2) is still a debated open 
issue. 

Although the configuration space geometric properties of random finite-connectivity 
spin glass models at or after the clustering transition have been well characterized by 
statistical physics methods, much less is known about the configuration space structure 
just before the clustering transition. Why do an exponential number of Gibbs states 
suddenly appear at the clustering transition? Are they preceded by precursor structures 
in the ergodic phase of the configuration space? If yes, when do the Gibbs-state 
precursors start to form and how to describe their evolution? What are the impacts 
of these precursor structures to the equilibrium dynamical properties of the system? 
These questions are important for a full understanding of the structural evolution of the 
configuration space. 

In this paper, as a continuation of our recent efforts [5j EJ El |8], we study the 
fine structure of the configuration space of random finite-connectivity spin glass models 
in the vicinity of the clustering transition. We focus on the evolution of the ground- 
state configuration space (corresponding to T = 0) using the constraint density a as 
the control parameter. As demonstrated through two case studies, a heterogeneity 
transition occurs to the ground-state configuration space at certain critical value of a. 
At the transition point, exponentially many communities of configurations emerge from 
the ground-state configuration space. Each configuration community is a collection 
of relatively similar configurations and it forms a stable thermodynamic phase in the 
presence of a suitable external field; they are the precursors for the Gibbs states at the 
clustering transition. The entropy density s(q) of configuration-pairs as a function of 
configuration-pair overlap q is calculated by the replica-symmetric (RS) and the first- 
step replica-symmetry-broken (1RSB) cavity methods. Extensive numerical simulations 
are performed to confirm that, the entropic trapping effect of ground-state configuration 
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communities leads to strong dynamical heterogeneity of diffusion processes within the 
configuration space. 

Dynamical heterogeneity in the fully-connected p-spin interaction spherical model 
(with continuous spins) [9] was quantitatively investigated by Donati and co-workers [10] 
within the framework of the Franz-Parisi effective potential theory [TH [TJ] • The results 
of the present paper for finite-connectivity systems with discrete spins are qualitatively 
similar to the results of [10]. Some aspects of the heterogeneity transition in random 
finite-connectivity systems are also shared by real-world D-dimensional spin glass and 
structural glass systems, such as hard-sphere poly-disperse colloidal systems, where the 
particle volume fraction plays the role of the constraint density a [10J. In real- world 
glass systems and supercooled liquid systems, dynamic heterogeneity occurs in real 
space. The present work is also related to the work of Krzakala and Zdeborova on the 
adiabatic evolution of a single Gibbs state of a finite-connectivity spin glass system as 
a function of temperature T [131 E] • 

Two model systems are studied in the paper. Section [2] concerns with the p- 
body spin glass system, which is equivalent to the random A'-XORSAT (exclusive- 
or-satisfiability) problem of computer science. Replica-symmetric and 1RSB mean-field 
calculations are carried out to obtain the entropy density of ground-state configuration- 
pairs. Section [3] focuses on the heterogeneity transition and the dynamical heterogeneity 
of the random A-satisfiability (if -SAT) problem. Results obtained by RS calculations 
and numerical simulations are reported in this section. We make further discussions in 
section HI 

2. The random p-body spin glass model 

The random p-body spin glass model is defined by the energy function 

M 

E psp in(a) = - ^2 Ja II 0~i , (1) 

a=l i£da 

where a = (cri,o"2, . . . ,o~n) denotes a spin configuration for the N vertices i G [1, A 7 "], 
with each spin variable <jj G {— 1,+1}; the index a G [1,M] denotes one of the M 
interactions of the system and J a is the quenched random coupling constant, whose 
value is fixed to J n = +1 or J„ = -1 with equal probability; the set da includes all 
the vertices that participate in the interaction a, its size is fixed to p, and each of its p 
different elements is randomly and uniformly chosen from the whole set of N vertices. 
Similar to da, we denote by di the set of interactions that involve vertex %. While each 
interaction a in the p-body spin glass model affects the same number p of vertices, the 
sets di may have different sizes for different vertices i. Actually, when N is large enough, 
the probability that a randomly chosen vertex participates in k interactions is governed 
by the Poisson distribution Pp(k) = e~ c c h /k\, with mean value c = pM/N. 

In the p-body spin glass model (CQ), each interaction contributes either a positive 
energy +1 or a negative energy —1 to the total energy. If we set p = K, this model is 



Ground-state configuration space heterogeneity of random spin glass models 



4 



equivalent to the random i-T-XORSAT problem with the energy function 




(2) 



For the random i^-XORSAT problem, each interaction a is also referred to as a 
constraint, whose energy (1 — J a Ylieda a i)/^ * s either zero (constraint being satisfied) or 
unity (constraint violated). The energy E xorsat (a) counts the total number of violated 
constraints by the spin configuration a. The constraint density a of the system is by 
a = M/N. 

The random i^-XORSAT problem (or equivalently, the random p-body spin glass 
model) has been well studied in the statistical physics community. It serves as an 
interesting system for understanding the low-temperature equilibrium property of finite- 
connectivity spin glasses [TJ [151 EH EE] , and for understanding the dynamical property of 
glassy systems [18j[T9]. This model is also closely related to error-correcting code systems 
of information science, such as the Sourlas code [20]. The ground-state configuration 
space structure of the random if-XORSAT problem has been investigated in great detail 
[TBI liTl l2i~] and was found to depend only on K and the constraint density a in the 
limit of iV — > 00. 

For a given value of K > 2 there is a satisfiability threshold a s (K). When the 
constraint density a is below a s (K), the ground-state energy of model ([2]) is zero (the 
system is in the SAT phase), but it becomes positive when a > a s (K) (the UNSAT 
phase). A zero-energy spin configuration of model ([2]) is referred to as a solution, and 
all the solutions form the solution space S of this system. The solution space of a large 
random i^-XORSAT problem is non-empty only if its constraint density is in the range 
of a < a s {K). We have a s {2) = 0.5, a s (3) = 0.918, and a(4) = 0.977 [16]. 

Before the solution space of the random .fT-XORSAT (K > 3) becomes empty at 
a > a s (K), it experiences an ergodicity-breaking (clustering) transition at the threshold 
value a = ctd(K), with ctd(3) = 0.818 and a^(4) = 0.772 [16]. For a < ad(K), the whole 
solution space forms a single Gibbs state (the meaning of a Gibbs state is explained 
geometrically in the following subsection). On the other hand, for a > ad(K), the 
solution space is no longer ergodic but is composed of exponentially many solution 
clusters (Gibbs states), each of which containing exactly the same number of solutions. 
For ad(K) < a < a s (K), solutions from different solution clusters are separated by 
energy barriers that are proportional to the vertex number N |18j . 

2.1. Non-concavity of the entropy function and solution space heterogeneity 

The similarity of two solutions a 1 , a 2 in the non-empty solution space S of a random 
.fT-XORSAT problem can be measured by the overlap value 



We denote by Af(q) the total number of solution-pairs in the solution space S with 
an overlap value q. This number is exponential in N in the SAT phase a < a s (K). 



(3) 




Figure 1. Non-concavity of the entropy function s(q) as denned by ((4]) and the large 
scale organization of the solution space of a random constraint satisfaction problem. 
s(q) is a concave function of the solution-pair overlap q when the constraint density a 
is low enough (left-most panel); at a = a cm , s(q) starts to be non-concave, marking the 
formation of many solution communities in the solution space (second panel from left); 
then as a increases to a — ad, s(q) becomes non-monotonic, the solution communities 
separate into different solution clusters and ergodicity of the solution space is broken; 
the solution clusters shrink in size as a further increases, and eventually the number 
of solution-pairs with intermediate overlap values q 6 [91,92] will be zero and s(q) is 
only positive for q > 92 and q < q\ . 



Therefore in the limit of large N, an entropy density s(q) is defined as 

s (g) = Iln^(g) . (4) 

The entropy density function s(q) contains rich information about the structure of the 
solution space S (see figure [p. The shape of s(q) has several qualitative changes as 
the constraint density a increases. The first qualitative change occurs at a = a cm , 
where s(q) becomes non-concave. This concavity change corresponds to the formation 
of (exponentially) many solution communities in the solution space (the large-scale 
homogeneity of the solution space is then broken) [HI [7j. We refer to a cm as the 
heterogeneity transition point. 

We introduce the following partition function Z(x) for the solution space S [7\ 

N 

^es^es i=i Q 

where x is a coupling field between the solutions. When x is positive, solution-pairs 
with larger overlap values have larger weights in the partition sum (jSj). Under the field 
x, the mean solution-pair overlap value is q(x) = (l/N)d\nZ(x)/dx. For N — > 00, we 
see from fl5]) that, q(x) = arg max g (s(g) + xq). At x — 0, q(0) is equal to go, the most 
probable solution-pair overlap value of the solution space. 

If the entropy density s(g) is concave in g e [g , 1], then q(x) increases continuously 
with x for x > 0. However if s(q) is non-concave, because there exists a line of slope 
— x* which touches s(g) at two different points g = q a and g = g^ > q a (figured]), then 
the value of q(x) jumps from q a to g& at x = x* [SI E]- This discontinuity of q(x) reveals 



N(s(q) + xq) 



(5) 



Ground-state configuration space heterogeneity of random spin glass models 



6 



the existence of a field-induced first-order phase transition at x = x*. The solution- 
pairs exhibit two different levels of similarity. For x > x*, the partition function Z(x) 
are contributed overwhelmingly by (intra-community) solution-pairs with overlap values 
> qj, (overlap-favored phase), while for x < x*, Z{x) is dominated by (inter-community) 
solution-pairs with overlap values < q a (entropy- favored phase). The difference qb — q a 
increases from zero as the constraint density a exceeds the critical value a cm . At 
a = a cm , the solution space is in a critical state, where the boundaries between different 
solution communities are elusive, and the field-induced phase transition is second-order. 

The Hamming distance between two solutions a 1 and a 2 is defined as D(a l ,a 2 ) = 
X)£Li(l — ajaf)/2, which is related to the solution-pair overlap by D{B X , a 2 ) = (N/2)(l — 
q(a 1 , <? 2 )). The solution space S can be represented by a graph of nodes and edges. Each 
node of this solution graph denotes a solution, and an edge is linked between two nodes 
of the graph if the corresponding two solutions has a Hamming distance not exceeding 
a specified value D . For the random if-XORSAT problem at a < ctd(K), there exists 
a minimum value of Dq <C N such that all the nodes of the solution graph are in 
a single connected component [18]. We take this minimum value as our edge linking 
criterion. (For a > ctd(K), if D is not of the same order as N, the solution graph 
will be a collection of exponentially many disjointed components.) In the ergodic phase 
of a < ad{K), we may introduce two particles to the solution graph. Initially these 
two particles are residing on the same node, say a , of the graph. In case the particles 
are uncoupled, then each particle performs a random diffusion in the solution graph 
independent of the other: Suppose at time t the particle is at node a, then at the next 
time step it will, with probability kg/k maxi make a move to a randomly chosen nearest- 
neighbor of this node, where kg is the number of attached edges (the degree) of node a, 
and k max is the maximal node degree in the solution graph. In the case the particles are 
coupled by a field x, however, the particle diffusions are mutually influenced, and the 
visited node-pairs (say a 1 and a 2 ) are no longer uniformly distributed but are favored to 
more similar pairs by a factor of e xN i( al ' a2 )_ i n the thermodynamic limit N — > oo, when 
the coupling field x is larger than x*, then even at time approaching infinity, the two 
particles will still be diffusing in the neighborhood of each other and in the neighborhood 
of the initial node a . Such a strong memory effect at field value x > x* is a dynamical 
manifest of the existence of communities in the solution space. 

2.2. Annealed approximation for s(q) 

If one knows a solution a 1 = (a\,a\, . . . ,a\j) for the K-XORSAT system (TSJ), it is 
convenient to perform a gauge transform Oi ^— o~io\ to the spin value of each vertex i. 
Under this transform, (T5]) is simplified to 

a=l 

In this transformed system, all the coupling constants are positive (J a = +1), and 
the overlap of the transformed solution a with the reference solution cr 1 is q(a) = 
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Figure 2. The entropy density s(q) of solutions at an overlap level q to a reference 
solution for the random X-XORSAT problem with K = 3 and a = 0.6, 0.7, 0.8, 0.818 
(from top to bottom). Black solid lines and red dashed lines are results obtained by 
the annealed approximation ([8]). and blue dots are results of the replica-symmetric 
cavity method (dotted lines just for guiding the eye). When (JX2J) is valid, expression 
((U) gives the exact mean value of s(q), but if it is violated, (jHJ is only an upper-bound 
for the mean value of s(q). 



(l/N)J2i°'i- Equation (JSJ) is independent of the reference solution a 1 . This is a well- 
known property of the i^-XORSAT problem, namely its solution space S has the same 
local and global structure when viewed from any of its solutions. Because of this nice 
property, instead of calculating the solution-pair number Af(q), we calculate the number 
N\(q) of solutions which have an overlap value q with a reference solution a 1 . We denote 
the corresponding entropy density also as s(q), i.e., s(q) = (l/N)\nMi(q)- (With this 
slight abuse of notation, the solution-pair entropy density as defined by ([4]) is s(q)+so(a), 
where so(a) is the entropy density of the whole solution space at constraint density a.) 

The average value of M\{q) over the ensemble of random .fT-XORSAT problems 
with fixed vertex number N and constraint density a is 

/ N \ / 1 , „k \ aN 



For iV — > oo we obtain the following annealed approximation for s(q) as 



s ann (q) = = -_ In — ^-ln^- + aln . (8) 

For K — 3, the function s ann (q) is concave in q only for a < 0.577. For 
0.577 < a < 0.778, s ann (q) is non-concave in q but is still monotonic in the range 
q E [0, 1]; the monotonicity of s ann (q) in q G [0, 1] is lost when a exceeds 0.778 (figureEJ). 
The same qualitative results are obtained for the random i^-XORSAT problem with 
K > 4. 
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The annealed approximation s ann (q) is an upper bound of the entropy density s(q) 
for a typical random i^-XORSAT problem. These two quantities are identical only when 
J\fi(q) has the self- averaging property, i.e., the value distribution of A/i(g) among the 
ensemble of random .fT-XORSAT problems of constraint density a approaches a delta 
function in the limit of iV — > oo. To check whether this self-averaging property is valid, 
we need to calculate the mean value of M 2 (q). After some combinatorial analysis, we 
obtain 



q)N/2 



where Q n = [1 + 2q K + (1 — An/N) K ]/A. In the N — > oo limit, we therefore have 



— lnA/?(g) = max s 2 (q,p), (10) 

N <p<min(i^,i±2) 

with s 2 (q, p) being expressed as 

1 + g l + q l-q l-q Ap 2 
s 2{q, P) = — In — — m — p In 



2 2 2 2 l-q 

1 



l + 2q K + (l- Ap) K 
+ am . 11 



Self-averaging of M\ (q) requires that 

A= max s 2 (q,p) - s 2 ann (q) = . (12) 

0<p<min(i T i,i±^) 

Numerical calculations reveal that ( 1T2|) is satisfied only for values of q very close to or 
very close to 1. In figure [2j s an n(q) is plotted as a black solid line if A < 10" 3 and as a 
red dashed line if A > 10~ 3 . 

As s an n(q) is only an upper bound to s(q), the fact that s ann (q) becomes non- 
concave at a < aa(K) can not be used as a proof that s(q) also becomes non-concave 
at a < ctd(K). We proceed to calculate s(q) by the cavity method of statistical physics. 



2.3. Replica- symmetric mean-field analysis 

For the gauge-transformed if-XORSAT system (Q, if vertex % is involved in constraint 
a, we define a cavity probability pf_^ a as the probability that takes the value crj = +1 
when the constraint a is absent. For each constraint a we exploit the Bethe-Peierls 
approximation [U [221 E3] and assume that, the spin states of the vertices j G da are 
mutually independent in the absence of a. Under this approximation, we obtain that 
if the vertex i 6 da takes the spin value <jj, the probability that constraint a being 
satisfied is equal to [1 + o~i Y[jeda\i(^Pj^a ~ l)]/2, where da\i means the subset of da 
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Figure 3. The mean overlap q(x) for the random 3-XORSAT problem at a = 0.617, 
0.618, 0.619, 0.620 (from right to left). For a < a cm {K = 3) = 0.618154, q(x) is a 
continuous function of x. At a = a cm (3), the slope of q(x) diverges at x = 0.151473. 
For a > a cm (3), q(x) is discontinuous and has a hysteresis loop. 



that is missing element i. Then the following belief-propagation equation can be written 
down for each vertex-constraint association (i,a): 



pUa = P({pUb}) 



e x n 

b£di\a 


p+ n (2p;_ >b -i)- 

j£db\i 

2 




e x n 

b£di\a 


p+ 11 (atf-i-i)" 

j£db\i 


+ e~ x n 

bGdi\a 


p- n (^pU b -m 

j€db\i 


2 


2 



(13) 



For a given energy function (|6]), there are M x K iteration equations, which form the 
replica-symmetric cavity theory [U [22], [23] . 

After this set of belief-propagation iteration equations has reached a fixed point, 
the mean overlap with the reference solution is calculated as 



1 N 

= J* E 

i=l 



e x n 


_i + n (SP+^-l)" 


-e- a n 


p- n ( 2 ^a-i)i 

j£da\i 


2 


2 


e x n 

addi 


"i+ n (2p^.-i) _ 


+ e-^ n 

addi 


n (2p+_ a -i)- 

j<E.&a\i 


2 


2 



■ (14) 



And the entropy density function of x is 



1 N 

s ( x ) = T?E ln 
iV i=i 



e X I[ 

aG<9? 



i+ n (2pU a -i; 



j£da\i 



e -n 



i- n (2p+^-i) 



J/ 



/i+ n (2^+ -i: 



•— V(K- l)ln 



xq(x) . 



(15) 
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The mean value of s(x) as averaged over the ensemble of random .fT-XORSAT 
problems (at fixed value of a) can be calculated using the population dynamics technique 
[HET]- By eliminating x from s(x) and q(x) we obtain the entropy density function s(q). 
The numerical results are shown in figure [2] (blue dots) for K = 3 at different values of 
a. The mean overlap function q{x) is shown in figure [3]for K = 3 and a ~ 0.62. Similar 
results are obtained for the cases of K > 4. 

For the random 3-XORSAT problem, q(x) is a continuous and smooth function of 
field x when a < a cm (3) ~ 0.6182. As a approaches a cm from below, however, the 
maximal slope of q(x) is proportional to (a cm (3) — a) -1 and diverges at a cm (3). This 
divergence is a consequence of the fact that the entropy density s(q) starts to be non- 
concave at a = a cm (3). For a > a cm (3), q(x) as calculated by the RS cavity theory 
shows discontinuity and hysteresis behavior when x is close to certain threshold value 
x*, indicating the existence of two distinct phases of the solution space as viewed from 
the reference solution (a 1 ). One of the phases contains solution a 1 and the other similar 
solutions, whose overlap with a 1 is larger than certain characteristic value ~ (q a + q^) /2, 
see figure [TJ We regard this phase as the solution community of solution a 1 . If we 
choose another solution outside the solution community of a 1 , we will find that this new 
reference solution is also associated with a different solution community. 

For a cm (K) < a < ad(K), the solution space of the random if-XORSAT problem is 
therefore formed by exponentially many solution communities. As the solution space is 
very heterogeneous at this range of a, the replica-symmetric mean-field theory probably 
is not sufficient to describe its statistical property. We now proceed to study the solution 
space heterogeneity using the 1RSB cavity theory. 



2-4- First-step replica- symmetry-broken mean-field analysis 

For the gauge-transformed model (jEj) under the coupling field x, to apply the 1RSB 
mean-field theory, the solution space is first divided into an exponential number of 
Gibbs states 7 [H I2T] . Each Gibbs state 7 represents a subspace 5 7 C S, and its 
partition function is defined as 

z i( x ) = X) ex p z E ff < • ( 16 ) 

We can then define a 'free energy' density fj(x) as / 7 = (1/-/V) In Zj(x). This free energy 
density is the sum of two parts, / 7 = s 7 + xq 1) where s 7 is the entropy density of Gibbs 
state 7 and g 7 is the mean overlap level of solutions in <S 7 to the reference solution a 1 . 
The total 1RSB partition function of the system is 

Z lRSB ^J2( Z X = T, eNmfl = [dfex V (N[Z(f)+mf]) . (17) 

77 

In the above equation, m is the Parisi parameter, and £(/) is the complexity, which 
measures the 'entropy density' of Gibbs states at the free energy density level / [Tl |2T]. 
If we set m — 1 in (1171) . each Gibbs state contributes a term e N f"< to Zirsb, and then 
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Figure 4. Results of the m — 1 1RSB mean-field theory for the random 3-XORSAT 
problem with a — 0.80. The complexity, the entropy density of solution communities, 
and the total entropy density are shown in the upper, middle, and lower panel, 
respectively, as a function of the overlap q. Two sets of 1RSB mean-field results 
are obtained in the population dynamics when different types of initial values are used 
for the population. At each value of q, the result with the larger value of complexity 
and the smaller value of community entropy density should be considered. The two 
vertical dashed lines mark the boundary between the replica-symmetric (RS) and the 
replica-symmetry-broken (RSB) region. 



Z\rsb is the total partition function of the system, provided that the complexity £(/) 
calculated at m — 1 is non-negative. The existence of a nontrivial 1RSB solution at 
m = 1 is also a signature of the instability of the RS theory of the previous subsection 
[21 [3]. We therefore set m = 1 in our calculations. The details of the 1RSB cavity mean 
field theory are presented in Appendix A, and here we discuss some of the numerical 
results obtained by population dynamics on the random 3-XORSAT problem. 

At a < ct cm (3), no nontrivial 1RSB mean field results are obtained. The fixed 
point of the 1RSB population dynamics reduces to that of the RS theory for all the 
overlap values < q < 1. As a exceeds a cm (3), the replica-symmetric mean field theory 
becomes unstable for intermediate values of overlap q, and nontrivial fixed points of the 
1RSB mean-field population dynamics are observed. For example, at a = 0.80 we find 
that the complexity S(g) is exactly zero for overlap value q < 0.233 and q > 0.675, 
and S(g) is negative for 0.233 < q < 0.675 (see figure HJ). The fact that S(g) = for 
q > 0.675 suggests that, the solutions with overlap levels q > 0.675 to c? 1 are in a single 
solution community. In the corresponding solution subgraph of this solution community, 
any two solutions (nodes) with the same Hamming distance D to a 1 are connected by 
at least one path that involves only other solutions with the same Hamming distance 
D to a 1 (i.e., the subspace of solutions having the same overlap q (> 0.675) with a 1 is 
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Figure 5. Same as figure |H but for a = 0.81. The complexity E(g) is positive for 
0.21 < q < 0.425. 

ergodic within itself). 

The second message we get, from the fact that < for 0.233 < q < 0.675, 
is that the solution subgraph formed by all the solutions at the same overlap level q 
[G (0.233, 0.675)] to c? 1 is not ergodic within itself but is divided into greatly many 
disjointed connected sub-components, with a sub-exponential number of dominating 
ones. In this overlap range, the community entropy density and the total entropy 
density as obtained by the m = 1 1RSB mean-field theory can only be regarded as 
upper bounds for the true values. One needs to work with m < 1 to obtain better 
estimates for the community entropy density and the total entropy density. 

The third message we get from T,(q) = for q < 0.233 is that, the solutions 
with overlap levels to a 1 less than 0.233 form an ergodic subspace within itself. More 
precisely, all the solutions at each overlap level q (< 0.233) to a 1 are ergodic within 
themselves. The solutions in such a subspace of fixed overlap q come from different 
solution communities, but they are connected with each other in the solution graph 
even when only the edges inside the subgraph are remained. 

The theoretical results as shown in figure [5] and figure [6] for the random 3-XORSAT 
problem at a = 0.81 and a = 0.818, respectively, are similar with the results of figure HI 
At a = 0.81, the complexity S(g) calculated at m = 1 is positive for 0.21 < q < 0.425, 
indicating an exponential number of solution communities are equally contributing to 
the solution subspace at each overlap level q G [0.21,0.425]. For an overlap level 
q G (0.425, 0.69], however, the solution subspace is again dominated by a sub-exponential 
number of large solution communities. The situation at a = 0.818 is similar, but the 
range of q values for which S(g) > is further enlarged. 
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Figure 6. Same as figure gj but for a = 0.818 and a = 0.819. At a = 0.818, 
the complexity £(<?) is positive for 0.038 < q < 0.66 and is negative (w — 10~ 5 ) for 
0.66 < q < 0.705. At a = 0.819 > 0^(3), the complexity S(g) is positive even at q = 0. 



When the constraint density a exceeds the clustering transition point «d(3), the 
complexity E(g) at m = 1 becomes positive at q = (see the exemplar case of a = 0.819 
in figure [6]). This result means that the largest subspace of solutions (with overlap level 
q = to a 1 ) becomes non-ergodic at a > ad(3), as expected. 

2.5. Summary for K-XORS AT 

The calculations of this section demonstrated that the solution space of the random 
.fT-XORSAT problem experiences a heterogeneity transition as the constraint density 
a approaches a cm (K), where an exponential number of solution communities start 
to form in the solution space. These solution communities separate into different 
solution clusters at a larger constraint density value ad(K), where an ergodicity-breaking 
transition occurs. For a cm (K) < a < ad(K), although the solution space as a whole 
is ergodic, the subspaces of solutions with intermediate overlap levels to a reference 
solution are non-ergodic within themselves. 

Our theoretical results, combined with the results of |21j for a > ad(K), give a 
complete picture on the structural evolution of the solution space of the random K- 
XORSAT problem. 

3. The random /^-satisfiability problem 

The random K-SAT problem is a famous model system for the study of typical-case 
computational complexity of NP-complete combinatorial satisfaction problems [24J. Its 
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energy function E(a), like the random i^-XORSAT problem, is defined as a sum over 
M = aN constraints a: 

M 1 - JV 

m = e n • 

a=l igc9a ~ 

In (TT8|) . each constraint a affects a set da of randomly chosen vertices from the vertex 
set {1, 2, 3, ... , N}; J a is the preferred spin state of constraint a on the vertex i G da, 
it takes the quenched value +1 or —1 with equal probability. If at least one of the 
vertices i G da takes the spin value o~i = J a , the energy of the constraint a is zero, 
otherwise its energy is unity. The solution space S of model ( ITS"]) is formed by all the 
spin configurations a of zero total energy (i.e., satisfying all the constraints). In model 
f|T8|) . each vertex i is constrained by a set (denoted as di) of constraints. 

After the experimental demonstration of a satisfiability phase-transition in the 
random 3-SAT problem by Kirkpatrick and Selman [24J, studies on the solution space 
structure of the random i^-SAT problem have been carried out through rigorous 
mathematical methods (see, e.g., [251 ESI EZ]) and through statistical physics methods 
(see, e.g., [281 [291 EQl El EH 132] ) . The threshold constraint density a s (K) for the solution 
space to be empty is calculated to be 4.2667 for the random 3-SAT problem |30j and 
its values for K > 4 are also predicted by the 1RSB zero-temperature energetic cavity 
method [33]; a lower bound on a s (K) is calculated by the zero-temperature long-range 
frustration theory [Ml [E] . Ergodicity of the solution space is broken at the clustering 
transition point a = ctd(K) < a s (K) [291 EOj. At ad(K) < a < a s (K), the solution 
space contains an exponential number of Gibbs states. The value of otd{K) is calculated 
by the 1RSB zero-temperature entropic cavity method in [31 [35], reporting ad(3) ~ 3.87 
and a d (4) ~ 9.38. 

We are interested in the heterogeneity of the ergodic solution space at a < ad(K). 
In the following subsection we calculate the solution-pair mean overlap as defined by ([3]) 
using the replica-symmetric cavity method. The heterogeneity transition point a cm (K) 
is determined by this RS mean-field theory. 



3.1. Replica- symmetric mean- field analysis 



The partition function ([5]) is a weighted sum over all the solution-pairs (a 1 , a 2 ). Each 
vertex is then associated with a spin vector-state (a}, of). Under the coupling field x, we 
define the cavity probability p a ^i(a,a') as the probability that constraint a is satisfied 
if the vertex i e da takes the spin vector-state (a, a'). Similarly, we define the cavity 
probability p^ a (a, a') as the probability that vertex i is in the spin vector-state (a, a') 
in the absence of the constraint a G di. 



If we exploit the Bethe-Peierls approximation as mentioned in section 12. 3[ the 
following RS cavity equations can be written down for p a ^i and p^ a 



Pa- 
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+K J K' 1 II V^ai-Ji-Jl) , (19) 

j£da\i 

f a e w Il6eai\aPfr->i(ff,0 , onA 

Pi^a{(T, a = v ? ^ , 20 

E E e lffl<T » n 6 e9i\aP6->i(°'i ) °1) 

where 5^ is the Kronecker symbol (<5^ = 1 if m = n and 5^ = if m ^ n). There is a 
symmetry requirement for Pi^ a (a, cr'), namely that p^ a (+l, —1) = p^ (— 1, +1). This 
condition is a result of the fact that, the solution-pair (cr 1 , a 2 ) has the same contribution 
to the partition function (J5J) as the solution-pair (a 2 , a 1 ). 

After a fixed-point has been reached for the RS iterative equations, the mean value 
q(x) of the solution-pair overlap and the solution-pair entropy density s(q) as defined 
by OH) can then be calculated. For example, the mean solution-pair overlap is expressed 
as 

= n 2>* a i h = »f E v v — a — u „n > ( 21 ) 

7V i=i iV i=i E CTl E^ e ' 'HoeaiPo-^tV^ii^J 

where (. . .) ffi means averaging under the coupling field x. 

The predicted mean overlap function q(x) for the random 3-SAT problem is shown 
in figure 0(a) for a in the vicinity of 3.75. Similar to the results of the random 3- 
XORSAT problem shown in figure [3J the continuity of q(x) changes as a exceeds a 
critical value a cm (3) ~ 3.75. The jumping and hysteresis behavior of the mean overlap 
q(x) for a > a cm (3) indicates that solution communities start to emerge in the solution 
space of the random 3-SAT problem at a ~ 3.75. 

The overlap susceptibility x( x ) measures the sensitivity of the mean solution-pair 
overlap with the coupling field x, it is defined by x( x ) = dq(x)/dx. The susceptibility 
x( x ) is related to the overlap fluctuation by 

1 N N 

x( x ) = j: E E " (^>» W>»] • (22) 

Figure 0(b) demonstrates that, as a approaches 3.75 from below, the peak value of x( x ) 
becomes more and more pronounced and finally diverges. From the divergence of the 
peak value of x{ x )i we obtain that a cm (3) = 3.7497 for the random 3-SAT problem. 
This value is much below the value of «d(3) = 3.87. 

Similar mean-field calculations are performed for the random 4-SAT problem and 
we obtain that a cm (4) = 8.4746. This value is again much below the clustering transition 
point a d (A) = 9.38. 

3.2. Dynamical heterogeneity of Glauber dynamics 

The solution space heterogeneity of the random K-SAT problem influences the dynamics 
of random walking diffusion processes [6]. Similar to the random if-XORSAT problem, 
we can represent the solution space of the random i-T-SAT problem as a solution graph of 
nodes and edges, with the nodes denoting individual solutions and the edges connecting 



Ground-state configuration space heterogeneity of random spin glass models 



16 





Figure 7. Mean-field predictions on the mean solution-pair overlap q(x) (A) and 
the overlap susceptibility xi x ) (B) for the random 3-SAT problem. In (A) the 
constraint density a increases from 3.72 to 3.76 (right to left) with step size 0.01. 
The inset of (B) shows how the inverse of the peak value of x( x ) approaches zero as 
a increases to a cm (3) = 3.7497 from below. At a = a cm , x( x ) becomes infinite at 
x = x cm (3) = 0.00237. 



pairs of solutions of unit Hamming distance. When the constraint density a of the 
random K-SAT problem is less than aa(K), this solution graph has a single giant 
connected component that includes almost all the solutions of the solution space. In 
this ergodic phase of a < ctd(K), the structure of this huge solution graph become 
heterogeneous when the solutions aggregate into many different communities. There are 
many domains of high edge density in the solution graph, corresponding to the different 
solution communities. The nodes of different domains are also connected by many 
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edges, but the density of inter-domain edges is much lower than the density of intra- 
domain edges. As was demonstrated in [6], this heterogeneity of edge density causes an 
entropic trapping effect to diffusive particles on the solution graph. The dynamics of a 
diffusive particle can be decomposed into a trapping mode (the particle diffuses within 
a relatively dense-connected domain of the solution graph) and a transition mode (the 
particle escapes from one domain of the solution graph, wonders for a while, and then 
enters into another domain of the solution graph). As the trajectory of the diffusive 
particle oscillates between the trapping mode and the transition mode, if a clustering 
analysis is performed on a set of solutions sampled from this trajectory at equal time 
interval, a clear community structure can be observed among the sampled solutions 

The solution space diffusion process can be turned into a stochastic search 
algorithm. One such algorithms, the SEQSAT of [36], constructs a solution for a random 
A-SAT problem in a sequential manner. Constraints of the problem are added one after 
another in a random order, and as a new constraint is added, a random walk process of 
single-spin flips is performed in the solution space of the satisfied sub-problem to reach 
a solution that also satisfies the new constraint. It was observed [361 E] that, when the 
constraint density a of the satisfied sub-problem exceeds a cm (K), the SEQSAT search 
process becomes viscous, the mean waiting time needed to satisfy a new constraints 
starts to increases rapidly with a, and the sequence of waiting times starts to have large 
fluctuations. This dynamical behavior is easily understood in terms of the heterogeneity 
of the underlying solution space. 

Solution space structural heterogeneity results in dynamical heterogeneity of 
diffusion processes. To demonstrate this point more clearly and to estimate a typical 
relaxation time, we study in this subsection a simple solution space Glauber dynamics. 
For a large random A-SAT problem with N variables and M = aN constraints 
(a < ad(K)), first we construct through SEQSAT a spin configuration that satisfies all the 
M constraints. (Of cause we can also use other heuristic algorithms to generate an initial 
solution. The only requirement is that this solution should be a typical solution, or in 
other words, it should belong to the single largest solution cluster of the solution space. 
This requirement is satisfied in our simulation studies.) Then we set the initial time as 
t = —T e and denote the initial solution as 8(— T e ) = (ci(— T e ), o"2(— T e ), . . . , crjv(— T e )). 
The spin configuration is then updated by single-spin flips at each elementary time step 
At = 1/N. Let us suppose at time t the spin configuration is 8(t). Then a vertex i is 
chosen uniformly randomly from the whole vertex set {1,2, . . . , N}; a candidate spin 
configuration 8' is constructed, with a'j = Cj(t) if j ^ i and a[ = —ai(t). If 8' is not a 
solution of the A-SAT problem, then at time f = t + At, the old spin configuration is 
kept, i.e., 8(t') = 8(t). However, if 8' is also a solution, then with probability one-half 
8(f) = 8(f) and with the remaining probability one-half 8(f) = 8'. 

A unit time of the above-mentioned Glauber dynamics corresponds to N spin-flip 
attempts. The actual number of accepted spin flips in a unit time is about 0.1 A for 
the problem instances of figure [8] and figure M If this random walk diffusion process 
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Figure 8. Glauber dynamics simulation results for a single random if-SAT (K = 3) 
problem instance with N = 10 5 vertices. At a given value of a, the first M — aN 
constraints of this single random instance are used in the simulation. (Upper panel) 
Mean value of the overlap q(r) between two solutions separated by time r along the 
trajectory of configuration evolution. (Lower panel) The variance A(r) of the solution- 
pair overlap q(r). 



is simulated for an extremely long period of time, every solution in the connected 
component of the solution graph which the initial solution a(—T e ) belongs will have the 
same frequency of being visited. The time T e is set to be large enough (e.g., T e ~ 10 8 ) 
to ensure that the diffusion process has completely forget the initial solution <?(— T e ) at 
time t > 0. 

For the trajectory of spin configurations at t > 0, the quantity q(r) = 
(1/-/V) Y^Li o-i(t)ai(t + r) measures the overlap between two spin configurations a(t) and 
a(t + t) that are separated by a time r. q(r) is a random variable, its value fluctuates 
with different choices of the time t. We are mainly interested in the mean value q(r) 
and the variance A(r) of the overlap q(r). The mean overlap q(r) is calculated by 

1 N 

q(r) = (q(r)) t = - + r)>« , (23) 

iV i=i 

where (. . .)t means averaging over different starting times t > along the diffusion 
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Figure 9. Same as figure [SJ but for K = 4. 
trajectory. The variance A(r) is expressed as 

^ N N 

= m E E [(<n(t)<n(t + ^(^(t + r)) t - (o^a^t + r)> t <«r j (t)o- J -(t + r)) t ] . 
JV i=ij=i 

(24) 

Comparing ff2M with a dynamical susceptibility X4( T ) can be defined as X4( r ) = 
iVA(r) (such a quantity was introduced in [37J EE], see a l so the review [38]). 

The upper panel of figure [H] shows the relaxation behavior of q(r) for a large random 
3-SAT formula with iV = 10 5 vertices. To study how the shape of q{r) changes with 
constraint density a, the first M = aN constraints of this same formula is used for 
Glauber dynamics simulation at each value of a. We focus on a values in the vicinity 
of a cm (3). For a = 3.70, the mean overlap q(r) reaches its plateau value of 0.37 at 
t w 10 6 . For a = 3.75 ~ a cm (3), the plateau of q(r) is reached at r ~ 10 6,5 , and its 
value increases to 0.38. For a = 3.80, the plateau of q{r) is reached at r ps 10 72 , with 
a value of about 0.39. These and our other unshown simulation results clearly confirm 
that, as a exceeds a cm (3), the relaxation of the solution space diffusion process is slowed 
down greatly. 

The lower panel of figure M shows the variance A(r) of the solution-pair overlap 
values. At a = 3.70, the variance A(r) increases with r and reaches its plateau value 
at r w 10 6 . At a = 3.75, A(r) starts to show a peak at r w 10 5 - 5 . The peak of 
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A(t) becomes more and more pronounced as a further increases, and the time Tdh, 
corresponding to the peak value of A(r), shifts to larger values with a. At a = 3.80, 

T dh « 10 6 . 

The peak time t^h of the variance A(r) gives a measure of the typical time scale 
of dynamical heterogeneity of the Glauber dynamics. For r <C Tdh, two solutions a(t) 
and a(t + r) have a large chance of being in the same solution community, therefore 
the fluctuation of overlap values q(r) is small; a large fraction of vertices are inactive 
in this relatively short time window of r, as their spin values are flipped only with low 
frequencies. On the other hand, for r 3> r dh , the compared two solutions a{t) and 
a(t + r) have a large chance of belonging to two different solution communities, and 
therefore the fluctuation of their overlap values is again small; the spin values of most 
of the vertices have been flipped many times during such a large time window, and 
hence dynamical heterogeneity is destroyed. When the time window r is comparable to 
Tdh, the solutions a(t) and a(t + r) have comparable probabilities of being in the same 
solution community and being in two different communities; this causes relatively large 
fluctuation of the overlap values. As a increases from a cm (3) further, it takes more 
time for the diffusion process to escape from a solution community, and the difference 
between intra- and inter-community overlap values become larger, these two facts make 
the peak value A(r) and the peak time Tdh both to increase rapidly. 

Figure [9] reports the simulation results for a random 4-SAT problem instance of 
N = 10 5 vertices and M = aN constraints. Similar to the case of random 3-SAT, we 
find that the viscosity of the Glauber dynamics increases rapidly with constraint density 
a. For a = 9.10 > a cm (4), the mean overlap q(r) reaches its plateau value only at a 
very large value of r w 10 s . As demonstrated by the variance A(r) of solution-pair 
overlaps, dynamical heterogeneity become to manifest itself at a = 8.50 ~ « cm (4) and 
it becomes more and more pronounced as a further increases. 

Comparing figure M and figure [9] we can also notice an important dynamical 
difference between the random 3-SAT and the random 4-SAT case. For the random 
4-SAT system, we observe that the relaxation curve of g(r) is 'Z' form in shape at 
a = 9.10 (i.e., 97% of a<i(4)), with a plateau value of q ~ 0.7 at intermediate time 
intervals r before it finally decay to a much lower q ph 0.3 at r ~ 10 8 . The peak of the 
overlap variance A(r) corresponds to the time interval r at which q starts to decay from 
the larger plateau. However, for the random 3-SAT system, we find that the curve of 
q(r) has a 'L' shape even at a = 3.80 (98% of a d (3)), and at the time when A(r) reaches 
its peak value, the mean overlap g(r) is already close to its asymptotic value of r — > oo. 
We suggest that this dynamical difference is a strong reflect of the difference between the 
community structures of the random 3-SAT and the general random .fT-SAT problems 
with K > 4. 

For the random 3-SAT problem, in the heterogeneity phase of a cm (3) < a < a d (3), 
the solution space probably is dominated by a sub-exponential number of largest solution 
communities. Because of this predominance, the mean intra-community solution-pair 
overlap of each of these dominating communities is only slightly higher than the mean 
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solution-pair overlap of the whole solution space. Then at typical time t^h when the 
diffusive particle jumps between different dominating solution communities, one will not 
observe much drop in the mean overlap value q. Such a picture is consistent with the 
prediction that, at a > a^(3), only a sub-exponential number of solution Gibbs states 
dominate the solution space [5]. 

For the random i^-SAT problem with K > 4, however, in the range of a G 
(a cm (K), ad(K)) the solution space probably is contributed mainly by a exponential 
number of median-sized solution communities. The sub-exponential number of the 
largest solution communities has only a negligible contribution to the whole solution 
space and therefore barely influence the dynamics of the diffusive particle. For such 
a community structure, then the mean intra-community solution-pair overlap will be 
much larger than the mean solution-pair overlap of the whole solution space, resulting 
in a change of trend of q(r) at r ~ r^h- Such a picture for K > 4 is again consistent with 
the prediction that, at a > a^{K) the solution space is dominated by an exponential 
number of median-sized Gibbs states [3] . 

3.3. Summary for K -SAT 

In this section, we confirmed that the solution space of the random .fT-SAT problem 
becomes heterogeneous at a > a cm (K) and determined by the replica-symmetric cavity 
method that a cm (3) ~ 3.75 and a cm (4) ~ 8.47. We demonstrated that the existence 
of many solution communities in the solution space caused heterogeneous behavior in 
the dynamics of a solution space diffusion process. The typical time scale of dynamical 
heterogeneity of this diffusion process is determined by computer simulations. 

4. Outlook 

A heterogeneity transition was found to occur in the ground-state configuration spaces of 
two multiple-spin interaction systems, the random i^-XORSAT problem and the random 
K-SAT problem. We expect that this transition is a general phenomenon that occurs in 
many other spin glass systems before the ergodicity-breaking transition of the ground- 
state configuration space. Such a heterogeneity transition is unlikely to be special to 
the ground-state configuration space but should also be observed as the energy level (or 
equivalently the temperature T) of the configuration space is lowered to certain critical 
level. If the configuration space of a spin glass system is ergodic but highly heterogeneous 
at certain temperature T, this structural heterogeneity probably will manifest itself 
through heterogeneous behaviors in various spin relaxation dynamical processes of the 
system. More deep understanding on the relationship between the phenomenon of 
dynamical heterogeneity and the structural heterogeneity of the configuration space 
is very desirable. Such efforts may bring new ways of probing configuration space 
heterogeneity from observing features of dynamical heterogeneity. 

The solution space heterogeneity of the random K-SAT problem has been studied 
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analytically only through the replica-symmetric cavity method. However, from the 
experiences gained on the random A-XORSAT problem, we believe the replica- 
symmetric cavity theory is not sufficient for a heterogeneous solution space. A complete 
study on the heterogeneity transition of the random A-SAT using the 1RSB mean-field 
cavity theory will be reported in a later paper. 

In the case of the random A-SAT problem, we have not yet performed a systematic 
investigation on the scaling behaviors of the peak value of overlap variance (equivalently, 
the overlap susceptibility x( T )) an d the typical time Tdh of dynamical heterogeneity. 
Probably both the peak value of x( r ) an d the characteristic time t^h diverge at the 
clustering transition point a = ctd(A). To get unambiguous results, we need a more 
efficient protocol of simulating the diffusion dynamics on the solution space. 

Simulations results of [6j and [5] indicated that, for the random 3-SAT and random 
4-SAT problem, the single solution space Gibbs states at a > «d(A) are themselves 
very heterogeneous in internal structure. To study analytically the heterogeneity of 
single solution clusters in the ergodicity-breaking phase, however, appears to be a very 
challenging task. Such kind of investigations may be very valuable for us to understand 
how the largest solution clusters of the random A-SAT problem evolve with constraint 
density a. 
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Appendix A. The 1RSB cavity equations for the random A-XORSAT 



Here we list the technical details of the 1RSB mean-field cavity theory for the random 
A-XORSAT problem as discussed in section 12.41 For a comprehensive description of 
the 1RSB mean- field theory, the reader is referred to [U [21 (3j [35] . 

When there are an exponential number of Gibbs states, the probability pf^ a as 
defined in section 12731 will be different for different Gibbs states. The distribution of pf_+ a 
among all the Gibbs states is denoted by Pj_ >a (p+ >a ). We define pf^ a as the average 
value of pf_^ a , i.e., pf_^ a = J dp + Pi^ a (p + )p + . We also define two auxiliary distributions 
as [2] 



problem 



Qi-+a(pUa\PLa) 



Qi^a{Pi^-a\Pi^a) 




(1-pUa) 



(A.l) 



(A.2) 
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Qf-^ a (Pi^a\pf->a) * s the conditional probability of pf^ a given that o~i = +1. Similarly, 
Qi^ a (,Pi->a\pt->a) is the conditional probability of p^\. a given that o~i = —1. 

At the special case of m = 1, it can be shown that the iteration equation for pf_^ a 
has the same form as (|T3|) , with the only difference that all the pt^.1, values are replaced 
by their corresponding mean p~L>. 6 , i.e., p~t->a = PdP^b}) P]- The iteration equations 
for the conditional probabilities QU a (pt>a\pUa) and Qi^a(pUa\p~Ua) at m = 1 have 
the following expression [21 [3] 
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where cr^j denotes a spin configuration for the vertex set db\i of constraint 6, and 
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(A.4) 



is the probability of a satisfying spin assignment crg^i for constraint 6 given the spin 
value <Tj of vertex i (the probability pj^ = 1 — * s the mean probability of vertex j 
taking spin value Oj = — 1 in the absence of constraint 6). 

The cavity iterative equations for pf_^ a and Qi\ a (pt^a\pi-+a) can be solved by the 
population dynamics technique PQETJ- We are interested in the solution space property 
at a fixed value of the overlap value q, so in the population dynamics simulation the 
magnitude of the coupling field x is adjusted from time to time to ensure that the mean 
overlap value as expressed by ( TT4|) with p^ a replaced by Pjl+ a is equal to the specified 
value q (see, e.g., [3"5| I2"T]). 

The grand free-energy density g of the model system, as defined by g = 
(1/iV) hxZmsB, has the following simplified expression at m — 1: 
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The mean free energy density / of a Gibbs state is expressed as 

i JV i aN 
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where A fa and A/ a are, respectively, the free energy increase caused by vertex i and 
constraint a. These two free energy increases are expressed by the following expressions 
at m = 1: 
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°"aa\i jeda\i 



X 



in[E^n 



(A.7) 
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and 

In ( 1A.7P ). pf is expressed as 



W a {0- da ) II QiUa(pUa\Pi 
idda 



111 



'l + Uieda(2pL a -l) 



(A.8) 



Pi 



e x n 


"i+ n (2?^.-i)' 
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e x n 

addi 


"i+ n (2^.-1)' 

j€zda\i 


+ 6-^ n 

a£c9i 


"i- n (^„-i)" 
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2 



(A.9) 



and Pj = 1 — The probability w(aQ a ) in (IA.8I) is expressed as 

II Pi^ a $h a . 

idda iUea a ^ 



w a {o- da , 



E n ftUJh a . ' 
OBn iGda lli£a 1 



(A.10) 



At m = 1, the complexity S, the total free entropy density s to tai, and the mean 
entropy density of a solution community, s cm , are calculated to be 

^ a - f . (A.ll) 

(A.12) 
(A.13) 



9- f , 
Stotai = 9 - xq 



The equality s tota i = s cm + S holds at m — 1. 

It appears that at some overlap values q, more than one mean-field 1RSB solutions 
can be produced by the population dynamics simulation at m = 1. These different 
mean-field solutions have the same value of s to tai but different values of S and s cm . 
In such a case we choose the mean-field solution with the largest value of S (similar 
situations of multiple mean-field solutions were observed earlier in the random i^-SAT 
problem at m < 1 [4T)| [35] ) . 
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